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ABSTRACT 


Unlike  computer  based  instruction  of  knowledge  systems, 
instructional  feedback  for  dynamic  skill  training  has  been  found  to 
be  most  effective  when  the  student  chooses  when  and  if  feedback  is 
to  be  received  (Munro,  Fehling,  Blaise,  &  Towne,  1981).  Because 
students  in  dynamic  skill  training  are  often  heavily  loaded  with 
processing  demands,  instructional  feedback  must  be  postponed  until 
students  have  sufficient  free  resources  to  process  it.  The  present 
study  attempts  to  replicate  these  findings  using  a  simpler  task. 

The  second  factor  in  the  present  study  is  the  effectiveness  of 
computer  generated  voice  output  in  instruction  and  simulation  in 
dynamic  skill  training.  These  hypotheses  were  tested  in  an 
experiment  in  computer  based  instruction.  Both  the  intrusiveness 
and  delivery  mode  (text- voice)  factors  had  statistically 
significant  effects  on  student  errors.  The  group  which  performed 
the  best  received  feedback  in  a  textual  mode  and  had  control  over 
when  and  if  they  were  to  receive  feedback.  The  second  best  group 
received  feedback  in  a  computer  voice  mode  and  had  control  over 
when  and  if  they  were  to  receive  feedback.  The  third  best  group 
received  immediate  feedback  to  errors  and  feedback  that  was  in  a 
textual  mode.  The  group  with  the  poorest  performance  received 
immediate  feedback  to  errors  and  feedback  that  was  in  a  computer 
voice  mode.  The  results  suggest  ( 1 )  that  Instruction  in  dynamic 
skill  should  be  non- intrusive,  and  (2)  that  current  inexpensive 
voice  synthesis  technology  is  not  appropriate  for  dynamic  skill 
training. 
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INTRODUCTION 

Dynamic  skill  training  presents  problems  and  opportunities  for 
computer  based  instruction  (CBI)  that  are  not  present  in  knowledge 
system  training.  One  such  issue  is  how  instruction  given  in 
response  to  student  actions  cam  be  presented  most  effectively.  In 
conventional  CBI,  such  instructions  are  typically  presented 
immediately  after  the  student  response  that  evokes  them.  In 
dynamic  skill  training,  immediate  presentation  of  instructions  is 
termed  intrusive.  Because  students'  processing  resources  are  more 
likely  to  be  heavily  loaded  at  the  time  of  the  presentation  of  the 
instruction,  both  the  simulation  practice  task  and  student 
attention  to  the  instruction  are  likely  to  suffer. 

Previous  research  (Munro,  Pehling,  Blaise,  &  Towne,  1981)  has 
shown  that  students  who  can  determine  if  and  when  they  will  receive 
Instructional  feedback  messages  —  termed  non-intrusive  instruction 
—  make  fewer  errors  in  practice  than  students  who  receive 
intrusive  feedback  Instruction.  One  of  the  purposes  of  the 
experiment  reported  here  is  to  discover  whether  this  effect  holds 
in  a  less  demanding  task  than  that  used  in  the  previous  study.  A 
simpler  version  of  the  experimental  task  used  in  the  previous  study 
was  developed.  This  task  retains  the  essential  structure  of  the 
Air  Intercept  Controller  task,  while  imposing  fewer  requirements. 

The  Air  Intercept  Controller  (AIC)  task  requires  the  student 
AIC  to  use  a  simulated  tracking  computer  station  to  monitor  and 
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track  controlled  and  enemy  aircraft,  to  compute  recommended 
headings  for  the  controlled  aircraft,  and  to  direct  the  controlled 
aircraft  to  intercept  and  destroy  the  enemy  aircraft.  The  student 
station  includes  two  display  screens,  one  representing  a  radar 
screen  and  the  other  the  display  console  of  the  tracking  computer. 
Student  input  is  by  means  of  a  joystick  and  11  specially  labeled 
keys  on  the  display  console.  The  task  requires  close  monitoring 
and  expeditious  responses  to  certain  events,  such  as  the  appearance 
of  a  new  blip  on  the  simulated  radar  screen.  In  addition,  the 
students  must  periodically  perform  certain  tasks,  such  as  checking 
the  fuel  status  of  the  controlled  aircraft. 

A  new  issue  of  concern  in  the  present  study  was  the 
consequences  of  computer  generated  voice  output  in  simulation  and 
instruction  in  dynamic  skill  training.  A  crucial  concern  was 
whether  currently  available  low  cost  voice  output  devices  could 
play  a  useful  role  in  dynamic  skill  simulation  training.  Many 
dynamic  skill  tasks  require  the  use  of  voice.  A  natural  approach 
to  computer  based  training  of  these  skills  is  to  make  use  of 
computer  generated  voice.  The  experiment  is  designed  to  compare 
the  use  of  voice  with  the  use  of  displayed  text  for  simulation  and 
instruction  in  the  AIC  dynamic  skill. 

Two  very  distinguishable  computer-generated  voice  output 
devices  were  used  in  the  voice  conditions  of  the  practice  training. 
A  device  employing  a  pre-recorded  digital  representation  of  actual 
human  speech  was  used  to  simulate  the  responses  of  the  pilots  of 
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controlled  aircraft.  A  text-to-speech  synthesis  device  was  used  to 
deliver  the  sane  instructional  messages  sent  to  the  text  group 
students.  The  speech  quality  of  this  device  was  much  less  natural 
sounding  than  that  from  the  device  simulating  the  pilots'  voices. 

The  results  expected  from  this  experiment  were,  first,  that 
the  phenomenon  of  performance  decrements  due  to  intrusion  would  be 
replicated.  This  result  was  expected  to  obtain  despite  the  less 
strenuous  task  requirements  of  the  revised  AIC  task.  Second,  it 
was  expected  that  the  voice  conditions  would  be  superior  to  the 
text  version.  The  arguments  for  this  expected  result  were  that 
most  students  are  likely  to  be  better  at  listening  than  at  reading 
and  that  hearing  the  Instructional  messages  would  free  the 
students'  visual  attention  from  the  message  area  on  the  command 
console,  allowing  them  to  to  direct  it  to  the  task-oriented  areas 
of  the  console  screen  and  to  the  radar  screen.  It  was  expected 
that  presentation  mode  (voice  vs  text)  and  Intrusiveness  of 
instruction  would  not  interact. 
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The  Experiment 

Method 

Subjects .  Sixty- five  students  participated  in  the  experiment. 
Twenty-two  of  the  subjects  participated  in  the  experiment  to 
fulfill  an  introductory  psychology  course  requirement.  The 
remaining  forty-three  subjects  were  paid  volunteers  who  responded 
to  posted  notices  or  campus  paper  advertisments  at  the  University 
of  Southern  California.  Paid  subjects  received  $8.00  for  their 
participation  in  the  experiment.  Of  the  sixty-five  that 
participated  in  the  experiment,  sixty  completed  the  experimental 
training  task.  Two  of  the  non-paid  students  chose  to  discontinue 
the  training  task.  Poor  performance  of  two  others  required  that 
they  be  dropped  from  the  experiment.  A  temporary  equipment 
maladjustment  caused  one  subject  to  be  dropped. 

Procedure.  Subjects  were  run  individually  in  the  experient. 
Completion  of  the  training  session  required  from  one  hour  and 
forty-five  minutes  to  two  hours  and  forty  minutes.  All  subjects 
first  viewed  a  six  minute  videotaped  explanation  and  demonstration 
of  the  Air  Intercept  Controller  task.  Then  they  were  instructed  in 
the  functions  of  each  of  the  control  devices  used  in  the  simulated 
task—  eleven  specially  labeled  keyboard  keys  and  a  joystick— by  a 
computer-based-training  program  called  PREAIC.  The  PREAIC  program 
consisted  of  a  series  of  text  presentations  describing  the  task  in 
greater  detail  than  had  been  presented  in  the  videotaped 
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introduction.  It  also  presented  simulation  segments  with  which  the 
student  was  required  to  interact  by  using  the  control  keys  and 
joystick. 

After  completing  the  Pre-AIC  computer-based  intruction 
program,  all  students  then  viewed  the  same  videotaped  sequence 
which  review  the  special  keys  used  in  the  simulation,  and  required 
them  to  depress  each  key  as  it  was  reviewed.  At  this  point  the 
treatment  of  students  in  the  two  groups  diverged.  Each  group 
viewed  a  videotape  segment  describing  the  way  in  which 
instructional  feedback  would  be  presented  to  that  group  and  how 
they  should  respond.  This  segment  lasted  about  one  minute.  Next 
subjects  either  heard  or  viewed  each  feedback  message  to 
familiarize  them  with  the  advisories.  For  the  last  part  of  the 
introduction  subjects  viewed  a  videotape  of  how  to  perform  during  a 
practice  problem  for  their  particular  experimental  condition. 

Total  time  spent  on  the  Introduction  varied  from  thirty  minutes  to 
about  forty-five  minutes.  Students  were  then  given  practice  in  the 
Air  Intercept  Controller  task,  using  a  simulator  trainer  program 
called  AIC.  The  AIC  progrm  presented  a  series  of  20  problems  to 
the  student,  organized  in  three  banks  of  five,  ten,  and  five 
problems.  Difficulty  was  held  roughly  constant  within  each  bank, 
but  increased  with  the  progression  of  problem  banks.  Students  in 
all  four  conditions  received  the  same  problems,  and  the  training 
program  was  the  same  for  students  in  the  four  groups  in  every 
respect  except  intsructional  feedback. 
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Inatructional  Feedback  Treatments.  The  AIC  program 
continually  monitored  student  performance  for  a  variety  of  errors. 
Examples  include  inacurrately  positioning  a  symbol  on  the  simulated 
radar  screen,  or  falling  to  get  a  fuel  status  update  from  the 
pilots  of  the  simulated  aircraft  within  the  required  time.  The 
Appendix  contains  a  complete  list  of  these  errors.  For  all 
feedback  conditions,  when  the  AIC  program  detected  an  error,  a 
warning  tone  sounded  and  the  word  "Advisory"  appeared  in  an  area  of 
the  computer  console  display  reserved  for  instructional  messages. 

At  this  point,  those  students  in  the  intrusive  text  feedback 
group  were  presented  with  a  one-line  to  four-line  instructional 
message  related  to  the  error  just  detected.  While  the  message  was 
displayed,  the  simulation  was  frozen .  The  radar  screen  did  not 
change,  and  all  the  normally  active  keys  of  the  computer  console 
were  dead.  Only  one  key,  the  "Accept  message”  key,  was  active 
until  all  feedback  messages  were  seen  by  the  student.  After  the 
last  currently  active  message  was  seen,  the  word  "Advisory"  was 
erased  from  the  screen  along  with  the  last  message.  In  the 
intrusive  voiced  feedback  group  subjects  instead  heard  the  message 
via  headphones  while  the  simulation  was  frozen.  Subjects  were 
required  to  depress  the  "Accept  message"  key  until  all  advisory 
messages  had  been  heard.  Only  then  was  the  word  "Advisory”  erased 
and  the  simulation  continued. 

The  students  in  the  non-intrusive  text  feedback  group  were  not 
immediately  presented  with  the  instructional  message  after  the 
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system  sounded  the  error  tone  and  displayed  "Advisory”  in  the 
reserved  area.  Unlike  the  students  in  the  intrusive  conditions, 
students  in  the  non-intrusive  text  feedback  group  were  able  to 
choose  the  time  of  the  appearance  of  the  error  messages  by 
depressing  a  special  "Help”  key.  Depressing  this  key  caused  the 
error  message  to  appear  and  the  simulation  to  freeze  until  the 
student  pressed  the  "Accept  message"  key.  If  more  than  one  error 
had  been  detected  by  the  system  before  the  feedback  message  was 
requested,  then  the  most  recent  error  message  was  presented  to  the 
subject.  In  each  case,  depressing  the  "Accept  message"  key  caused 
the  error  message  to  be  erased  and  the  simulation  to  resume.  When 
all  the  pending  feedback  messages  were  presented,  the  word 
"Advisory"  was  removed.  If,  at  the  end  of  a  problem,  the  student 
had  not  viewed  messages  for  all  the  errors  detected  by  the  system, 
then  the  student  was  given  the  option  of  seeing  those  messages 
before  begining  the  next  problem. 

As  with  the  non-intrusive  text  group,  subjects  in  the 
non-intrusive  voiced  group  were  able  to  choose  the  time  of  their 
hearing  the  error  messages  by  depressing  the  special  "Help"  key. 

By  depressing  this  key,  subjects  were  able  to  hear  the  error 
message  while  the  simulation  was  frozen.  Simulation  resumed  when 
the  subject  depressed  the  "Accept  message”  key.  Only  when  all  the 
messages  had  been  heard  did  the  word  "Advisory”  erase  from  the 
screen.  As  with  the  proceeding  group,  if,  at  the  end  of  the 
problem,  the  student  had  not  heard  all  the  messages  for  the  errors 
detected  by  the  system,  the  subject  was  given  the  option  of  hearing 
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those  nessages  before  beginning  the  next  problem. 

In  summary ,  students  in  the  Intrusive  feedback  groups  were 
presented  with  an  error  message  for  each  detected  new  error  at  the 
time  that  the  AIC  program  recognized  the  error.  Subects  in  the 
intrusive  text  group  read  the  error  message  and  subjects  in  the 
intrusive  voiced  group  heard  the  message.  Students  in  the 
non-intrusive  feedback  groups  had  the  option  of  deteraining  when 
and  whether  they  would  receive  the  error  messages.  Subjects  in  the 
non-intrusive  text  group  read  the  messages  at  their  discretion 
while  subjects  in  the  non-intrusive  voiced  feedback  group  heard  the 
messages  when  they  wanted  to. 

Data  collection.  The  AIC  simulation  training  program 
preserved  an  exhaustive  record  of  each  student's  Interactions  with 
the  program.  These  data  sets  were  later  processed  by  data 
extraction  programs  to  produce  records  of  errors,  time  on  problems, 
and  other  variables  of  interest. 

Rggulta 

Errors.  Number  of  errors  was  used  as  one  measure  of  learning. 
Table  1  presents  an  analysis  of  variance  of  the  error  data.  The 
mean  number  of  errors  for  the  students  in  the  Intrusive  text  group 
was  97.9,  and  for  the  intrusive  voiced  group,  the  mean  number  of 
errors  was  143.8.  The  mean  number  of  errors  for  the  students  in 
the  non-intrusive  text  group  was  76.6,  and  for  the  non-intrusive 
voiced  group,  the  mean  number  of  errors  was  89.1 •  These 
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differences  were  highly  significant,  suggesting  that  students  in 
the  non-intrusive  groups  learned  more  than  the  intrusive  groups  and 
also  that  subjects  in  the  text  groups  performed  better  than 
subjects  in  the  voiced  groups. 

Time  per  problem.  The  total  time  spent  on  each  problem  by 
each  student  was  recorded.  Table  2  presents  the  analysis  of  the 
total  time  on  problem  data,  where  time  is  expressed  in  tenths  of 
seconds.  Intrusive  text  group  students  spent  a  mean  of  218.1 
seconds  per  problem,  and  the  intrusive  voiced  group  students  took 
241.3  seconds.  Non-intrusive  text  subjects  spent  an  average  of 
215.0  seconds  per  problem  and  non-intrusive  voiced  subjects  took 
230.2  seconds.  The  difference  was  not  significant  for  either 
factor.  Actual  time  spent  on  each  problem  was  also  recorded  for 
each  subject.  Actual  time  is  the  total  time  spent  on  a  problem 
minus  the  time  the  subject  spent  attending  to  feedback.  Table  3 
presents  the  analysis  of  the  actual  time  on  problem  data,  where 
time  is  expressed  in  tenths  of  seconds.  Intrusive  text  group 
students  spent  a  mean  of  201 .0  seconds  per  problem,  and  the 
intrusive  voiced  students  took  213-6  seconds.  Non-intrusive  text 
subjects  spent  202.5  seconds  per  problem  and  non-intrusive  voiced 
subjects  took  206.8  seconds.  This  difference  was  not  significant 
for  either  factor. 

Crucial  and  non-cruolal  errors.  Student  errors  are  classified 
by  the  AIC  program  into  twenty-eight  types.  Of  these,  eighteen  may 
be  termed  "crucial"  errors,  in  that  they  are  likely  to  materially 
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affect  the  student's  chances  of  "winning"  an  exercise  by  shooting 
down  the  enemy  aircraft.  The  other  ten  types  of  errors  are 
non-crucial  in  that  they  reflect  errors  of  form  that  will  not 
immediately  decrease  the  chances  of  winning  the  problem.  Table  4 
presents  the  analysis  of  total  crucial  errors  for  all  twenty 
problems.  It  shows  that  there  is  no  significant  difference  between 
any  of  the  conditions.  The  intrusive  text  group  made  an  average  of 
2.74  crucial  errors  per  problem,  while  the  intusive  voiced  group 
made  2.91.  The  non-intrusive  text  group  made  an  average  of  2.66 
crucial  errors  per  problem,  while  the  non-intrusive  voiced  group 
made  2.58. 

The  mean  number  of  non-crucial  errors  per  problem  for  the 
intrusive  text  group  was  2.12.  The  intrusive  voiced  group  made  a 
mean  of  4.24  non-crucial  errors  per  problem.  The  mean  number  of 
non-crucial  errors  per  problem  for  the  non-intrusive  text  group  was 
1.16,  while  the  non-intrusive  voiced  group  made  1.85  non-crucial 
errors  per  problem.  Table  5  shows  that  these  results  are 
significant.  This  suggests  that,  even  though  overloaded  by  the 
intrusive  instructional  messages,  the  intrusive  groups  were  still 
able  to  decide  which  performance  factors  to  attend  to.  They  chose 
to  permit  greater  deterioration  of  their  non-crucial  performance 
rather  than  their  crucial  performance.  It  also  suggests  that 
voiced  feedback  is  detrimental  to  performance,  particularly  when  it 
is  intrusive.  Subjects  that  receive  the  spoken  messages  are 
required  to  attend  to  the  error  message  for  a  much  longer  time  and 
this  requires  more  processing  than  text  subjects  who  simply  glance 
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at  the  written  error  message. 

Joystick  errors.  In  an  attempt  to  determine  what  kind  of 
performance  is  affected  by  the  Intrusiveness  and  mode  of 
instructional  feedback,  a  separate  analysis  of  joystick  errors  was 
performed.  Most  of  the  AIC  task  requires  the  fusion  of  skills  of 
planning,  time  or  distance  estimation,  and  decision  making,  as  well 
as  some  motor  coordination.  This  is  the  task  of  using  the  joystick 
and  keyboard  to  "hook"  a  symbol  on  a  simulated  radar  screen  blip. 
Table  6  shows  that  there  was  no  significant  difference  in  the  total 
number  of  joystick  errors  made  by  subjects  in  the  four  experimental 
conditions  over  the  course  of  the  twenty  practice  problems.  This 
implies  that  the  deleterious  effects  of  intrusive  and/or  voiced 
feedback  may  not  equally  degrade  all  types  of  skills.  The  motor 
skill  of  using  the  joystick  appears  not  to  be  harmed  by  the 
processing  loads  imposed  by  intrusive  or  voiced  feedback. 
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Discussion 

The  error  results  support  the  hypothesis  that  the  processing 
demands  of  dynamic  skill  simulation  training  require  non-intrusive 
rather  than  intrusive  feedback.  Students  receiving  intrusive 
feedback  made  significantly  more  errors  than  did  those  who  received 
non-intrusive  instruction.  Analysis  of  crucial  and  non-crucial 
errors  reveals  that  there  is  a  significant  difference  in  number  of 
errors  made  only  for  non-crucial  errors.  Apparently  because  the 
task  is  easier  than  that  used  in  the  previous  experiment  (Munro  et 
al,  1981),  intrusive  group  students  were  able  to  perform  the 
crucial  sub-tasks  as  well  as  the  non-intrusive  group  students. 

Only  on  the  non-crucial  subtasks  did  the  intrusive  group  students 
make  significantly  more  errors  than  did  the  non-intrusive  group 
students. 

The  separate  analysis  of  Joystick  errors  supports  the  view 
that  motor  skills  may  not  be  as  affected  by  the  information 
processing  overload  imposed  by  intrusive  instruction  as  are 
memory  and  decision-making  processes. 

Further  research  is  called  for  to  determine  whether  errors  in 
training  are  indicative  of  the  level  of  final  performance  after 
more  extensive  training.  If  students  in  the  intrusive  instruction 
group  make  more  errors  in  the  first  few  hours  of  training,  does 
this  mean  that  they  will  necessarily  make  more  errors  after  more 
exhaustive  training?  Will  they  reach  a  criterion  level  of 
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performance  more  rapidly?  More  exhaustive  longitudinal  studies  are 
called  for  to  ans wer  these  questions.  The  present  study  suggests 
that  if  an  objective  is  to  minimize  errors  during  dynamic  skill 
training,  then  non-intrusive  instruction  is  preferable  to  intrusive 
instruction. 

The  error  results  did  not  support  the  hypothesis  that  voice 
instruction  in  dynamic  skill  training  is  more  effective  than  text 
instruction.  To  the  contrary,  voice  instruction  resulted  in 
significantly  more  errors  than  did  text  instruction.  Two  plausible 
explanations  can  be  offered  for  this  result.  The  first  is  that  it 
takes  longer  to  listen  to  a  spoken  message  than  to  read  the  same 
message,  at  least  for  practiced  readers.  Therefore,  voice 
instruction  took  attention  away  from  the  ongoing  task  for  longer 
periods  of  time,  resulting  in  more  short-term  memory  decay  than  for 
those  students  who  read  the  same  messages.  The  second  explanation 
is  that  the  quality  of  the  voice  Instruction  was  not  very  good,  so 
that  listening  to  the  instructions  required  significantly  more 
processing  resources  than  reading  the  same  instructions. 

The  disappointing  performance  of  voice  instruction  in  this 
experiment  does  not  necessarily  mean  that  voice  cannot  be  used 
effectively  in  dynamic  skill  training.  It  does  mean  that  the 
low-quality  voice  output  used  in  this  experiment  may  not  be 
appropriate  for  dynamic  skill  training.  It  remains  to  be 
determined  whether  a  more  intelligible  voice  output  device  would 
cause  performance  decrements. 
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APPENDIX:  Crucial  and  Non-Crucial  Errors 


CRUCIAL  ERRORS 


1.  Hooking  symbol  off  the  blip.  This  error  occurs  when  the  student 
depresses  one  of  the  symbol  keys  (e.g.,  B1)  when  the  cursor  is  not 
positioned  directly  over  the  blip.  It  is  a  crucial  error  because 
subsequent  interception  computations  will  be  based  on  the  position 
and  heading  of  the  symbol  rather  than  the  blip. 

2.  Falling  to  hook  a  CAP  for  more  than  60  seconds  after  it  appears. 
This  is  the  error  of  not  identifying  the  new  blip  for  more  than  a 
minute  after  it  appears.  No  intercept  computations  can  be  made  for 
that  CAP  until  it  has  been  hooked. 

3.  Falling  to  rehook  the  CAP  for  more  than  30  seconds  after  first 
hooking  it.  This  error  occurs  when  the  student  identified  a  CAP  but 
failed  to  label  it  a  second  time.  The  tracking  computer  has  no  speed 
or  direction  for  the  CAP  that  was  not  rehooked. 

4.  Failing  to  hook  a  Bogey  for  more  than  24  seconds  after  it 
appears.  This  error  is  similar  to  2,  above. 

5.  Failing  to  rehook  a  Bogey  for  more  than  18  seconds  after  first 
hooking  it.  This  error  is  similar  to  3,  above. 

6.  Failing  to  rehook  a  Bogey  for  more  than  36  secods  after  a  heading 
Jink.  This  error  occurs  when  the  Bogey  changes  direction  from  the 
currently  plotted  path  and  is  not  rehooked  (relabeled).  The  tracking 
computer  will  use  the  old,  incorrect  course  for  any  computations  of 
intercept,  eto. 

7.  Failing  to  rehook  a  Bogey  for  more  than  36  seconds  after  a  speed 
jink.  This  error  occurs  when  the  student  fails  to  rehook  the  Bogey 
blip  after  it  speeds  up  or  slows  down.  After  a  speed  jink,  the 
Bogey's  radar  blip  does  not  match  the  position  of  the  Bogey's  label 
on  the  screen.  As  with  error  6,  subsequent  computations  based  on 
the  Bogey's  position  and  speed  will  be  incorrect. 

8.  Falling  to  rehook  a  CAP  for  more  than  36  seconds  after  it  does  a 
heading  Jink.  As  in  error  6,  a  change  in  direction  must  be  entered 
into  the  tracking  computer  or  subsequent  computations  will  be 
incorrect. 

9*  Falling  to  oompute  an  attack  heading  within  18  seconds  of 
rehooking  a  Bogey.  After  a  Bogey  has  been  hooked  and  rehooked,  the 
tracking  oomputer  has  a  representation  of  its  speed  and  direction. 

If  the  student  fails  to  oompute  an  intercept/attack  heading  quiokly, 
the  Bogey  may  escape  from  the  nearest  CAP. 

10.  Failing  to  oompute  an  attack  heading  within  18  seconds  of  a  jink 
by  a  Bogey  under  attack.  A  Bogey  being  attacked  must  not  only  be 


rehooked  quickly  after  it  jinks,  but  a  new  attack  heading  must  be 
computed,  as  well. 

11.  Falling  to  rehook  a  CAP  for  more  than  36  seconds  after  a  CAP 
makes  a  speed  jink.  As  with  error  7,  subsequent  computations  depend 
on  accurate  tracking  of  blips. 

12.  Falling  to  send  a  new  attack  heading  to  a  CAP  for  more  than  18 
seconds  after  it  spashes  a  bogey.  If  more  than  one  Bogey  is  to  be 
attacked  by  a  CAP,  then  a  new  attack  heading  must  be  sent  to  the  CAP 
to  direct  it  to  intercept  the  next  Bogey  after  it  destroys  the  prior 
assigned  Bogey. 

13*  Sending  an  incorrect  attack  heading.  This  error  occurs  when  a 
student  misreads  or  mistypes  the  previously  calculated  attack 
heading. 

14.  Falling  to  send  an  attack  heading  for  more  than  12  seconds  after 
computing  it.  The  computed  attack  heading  should  be  sent  quickly  to 
the  CAP  if  the  intercept  is  to  be  effected. 

15.  Failing  to  fire  while  in  firing  range.  The  CAP'S  weapons  are 
effective  only  at  close  range.  If  a  CAP  moves  into  firing  range  of 
its  assigned  Bogey  and  then  moves  out  of  that  range  again  without 
firing,  then  the  student  has  made  this  error. 

16.  Firing  when  not  on  attack  heading.  No  attack  heading  has  been 
sent  to  the  CAP,  so  the  fired  missle  is  wasted. 

17.  Falling  to  fire  for  more  than  12  seconds  after  entering  firing 
range.  Waiting  this  long  in  firing  range  is  likely  to  result  in 
being  shot  down  by  the  enemy. 

18.  Firing  when  not  on  correct  attack  heading.  If  a  CAP  fires 
when  it  is  not  on  a  oorrect  attack  heading  for  its  Bogey,  then  a 
missle  has  been  wasted. 


NON-CRUCIAL  ERRORS 

1.  Falling  to  rehook  a  CAP  for  more  than  36  seconds  after  it  turns 
to  attack.  This  error  is  non-cruclal  because  the  CAP  is  on  the 
correct  attack  heading  and  can  successfully  down  the  Bogey. 

2.  Firing  when  out  of  firing  range.  The  student  fires  a  missle 
before  the  CAP  is  dose  enough  for  its  weapon  to  be  effective. 

3*  Firing  when  out  of  missies.  This  is  non-cruoial  because  it  does 
not  affeot  the  number  of  Bogeys  shot  down. 

4.  Falling  to  get  a  fuel  and  weapons  update  for  more  than  60  seconds 
after  rehooking  a  CAP.  The  update  informs  the  student  how  many 


pounds  of  fuel  and  how  many  weapons  each  active  CAP  has.  This 
information  does  not  contribute  to  the  core  task  of  destroying  the 
Bogeys. 

5.  Failing  to  get  a  fuel  and  weapons  update  for  more  than  60  seconds 
after  the  previous  such  update. 


Total  errors,  means.  By  instructional  treatment  group. 

Intrusive  Feedback  Non-intrusive  Feedback 

Group  Group 

Text  97.9  76.6 

Voice  143.8  89.1 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significance 

Intrusion 

1 

12.40 

.001 

Delivery  mode 

1 

7.32 

.01 

Intrusion  x  Mode 

1 

2.39 

n.s. 

Residual 

56 

Table  1 . 


Total  Errors.  Analysis  of  Variance 
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Total 

crucial  errors,  means. 

By  instructional  treatment  group. 

Intrusive  Feedback 

Non-intrusive  Feedback 

Group 

Group 

Text 

54.93 

53.27 

Voice 

58.30 

51.70 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significance 

Intrusion 

1 

0.76 

n.s. 

Delivery  mode 

1 

0.04 

n.s. 

Intrusion  x  Mode 

1 

0 .27 

Residual  56 


Table  2. 


Crucial  Errors.  Analysis  of  Variance 
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Total  non-cruclal  errors,  means.  By  Instructional  group. 
Intrusive  Feedback  Non-intrusive  Feedback 


Group 

Group 

Text 

42.5 

23.33 

Voice 

84.9 

37.10 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significance 

Intrusion 

1 

21.68 

.001 

Delivery  mode 

1 

15.18 

.001 

Intrusion  x  Mode 

1 

3.95 

n.s. 

Residual 

56 

Table  3. 

Non-crucial  Errors.  Analysis  of  Variance 


Total  time  on  problems  (seconds),  means.  By  instructional  treatment  group. 


Intrusive  Feedback 

Non-intrusive  Feedback 

Group 

Group 

Text 

4362.2 

4300.0 

Voice 

4826.0 

4605.4 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significance 

Intrusion 

1 

0.91 

n.s. 

Delivery  mode 

1 

6.72 

n.s.1 

Intrusion  x  Mode 

1 

0.29 

n.s. 

Residual 

56 

•Approaches  significance,  <.02 


Table  4. 

Total  Time  on  Problem.  Analysis  of  Variance. 
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Actual  time  on  problems  (seconds),  not  including  feedback,  means. 
By  instructional  treatment  group. 


Intrusive  Feedback 

Non-intrusive  Feedback 

Group 

Group 

Text 

4020.8 

4050.2 

Voice 

4272.6 

4137.6 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significance 

Intrusion 

1 

0.22 

n.s. 

Delivery  mode 

1 

2.30 

n.s. 

Intrusion  x  Mode 

1 

0.54 

n.s. 

Residual 

56 

Table  5. 


Actual  Time  on  Problems.  Analysis  of  Variance 
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Joystick  errors,  means.  By  instructional  treatment  group. 


Intrusive  Feedback 

Non-intrusive  Feedback 

• 

Group 

Group 

Text 

19.33 

19.87 

Voice 

22.50 

17.60 

2-WAY  ANOVA 


Source  of  Variation 

DF 

F 

Significant 

Intrusion 

1 

1.57 

n.s. 

Delivery  mode 

1 

0.07 

n.s. 

Intrusion  x  Mode 

1 

2.42 

n.s. 

Residual 

56 

Table  6. 

Joystick  Errors.  Analysis  of  Variance. 
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